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To  summarize ,  progress  has  been  made  on  a  famcly  of  plated  problems, 

A  model  of  endstopped  visual  cortical  neurons  was  extended  to  include  compl 
components;  An  expensive  simulation  of  the  model  was  completed  with  regard  to 
orientation,  positional,  spatial  frequency,  curvature,  chevron,  and  end  line 
sensitivity;  Orientation  discontinuities  were  extended  into  the  motion  domain, 
and  -psychophysical  and  computational  experiments  were  performed  to  confirm 
hypothesis^  multiple  directions  being  represented  at  a  point  of  discontinu  y, 

A  theory  was  developed  to  capture  the  non-linearities  necessary  for  early 
measurement  of  orientation  and  curvature;  A  totally  different  theory  has  begun  to 
take  shape  for  functionally  characterizing  cytochrome  oxidase  blobs,  and  The 
mathematical  foundations  were  laid  for  a  theory  of  shape. 
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of  vision  sufficent  to  support  both  early  and  high  level  function;  i.e.,  to  start  with  images,  infer 
local  descriptions  of  their  geometric  structure  (both  in  space  and  in  motion),  to  group  these 
into  global  constructs  and,  ultimately,  to  recognize  the  objects  depicted  in  the  images.  Progress 
was  made  at  all  of  these  levels,  and  a  listing  of  AFOSR-supported  publications  attests  to  this 
activity,  The  logical/linear  machinery  has  also  been  applied  to  the  co-circularity  com;  r  r 
within  our  relaxation  network,  and  has  greatly  enhanced  its  selectivity  just  as  it 
operator  responses  above. 

The  development  of  logical/linear  operators  and  the  advancements  to  our  co-cir.j  v  , 
laxation  networks  also  enabled  us  to  achieve  another  of  our  design  goals:  the  representr. 
discontinuities  (e,g,,  corners  in  the  case  of  orientation)  in  early  vision.  The  representation  .• 
discontinuities  has  eluded  physiologists,  and  research  in  computer  vision  has  been  inconclusive 
That  is,  "optimal  corner  detectors"  have  been  proposed,  but  no  one  has  been  able  to  discriminate 
corners  from  smooth  points  of  high  curvature.  Such  differences  are  key  to  vision,  because  they 
facilitate  object  segmentation  (recall  that  whenever  one  object  occludes  another,  a  discontinuity 
is  created  at  the  points  where  their  bounding  contours  intersect).  We  have  proposed  that  such 
corners  (and  discontinuities  in  general)  are  represented  via  multiple  tangerts  at  the  same  position 
In  physiological  terms  this  amounts  to  several  cells  firing  simultaneously  in  the  same  hypercolumn 
signaling  different  orientations.  Standard  (linear)  orientation  operators  typically  select  for  only 
one  orientation  at  each  position.  The  non-linearities  in  our  logical/linear  operators,  however, 
enable  the  signaling  of  such  situations,  and  thus  we  are  able  to  detect  them.  Such  singularities 
were  also  found  to  be  important  for  motion  and  optical  flow  as  well. 

Finally,  we  have  made  progress  on  the  global  aspects  of  vision  as  well.  Cortical  hypercolumn¬ 
like  representations  are  local,  in  the  sense  that  they  represent  an  estimate  of  the  orientation  at  a 
point;  how  can  global  curves  be  inferred  from  these  local  representations?  We  have  discovered  a 
class  of  algorithms  for  computing  coverings  of  them,  in  parallal,  which  have  important  implications 
for  parallel  computation.  Also,  the  global  curves  that  we  find  are  sufficent,  in  many  cases,  to 
support  a  theory  of  shape  that  we  are  also  developing. 

To  summarize,  progress  has  been  made  on  a  family  of  related  problems,  including 

o  A  model  of  endstopped  visual  cortical  neurons  was  extended  to  include  complex  compo¬ 
nents; 

o  An  extensive  simulation  of  the  model  was  completed  with  regard  to  orientation,  positional 
spatial  frequency,  curvature,  chevron,  and  end-line  sensitivity; 

o  Orientation  discontinuities  were  extended  into  the  motion  domain,  and  psychophysical  and 
computational  experiments  were  performed  to  confirm  the  hypothesis  of  multiple  directions 
being  represented  at  a  point  of  discontinuity; 

o  A  theory  was  developed  to  capture  the  non-linearities  necessary  for  early  measurement  of 
orientation  and  curvature; 

o  A  totally  different  theory  has  begun  to  take  shape  for  functionally  characterizing  cytochrome 
oxidase  blobs;  and 

o  The  mathematical  foundations  were  laid  for  a  theory  of  shape, 
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During  the  past  3  years,  and  aided  by  the  support  of  AFOSR  Grant  89-0260,  we  have  begun 
to  evolve  a  rather  novel  perspective  on  how  early  visual  computations  should  be  constructed 
Theoretically  our  position  is  a  mixture  of  insights  from  mathematics  and  computation,  on  one 
hand,  and  physiology  and  psychophysics,  on  the  other,  In  practice,  our  laboratory  has  tak?n  these 
ideas  into  the  domains  of  robotics,  biomedicine,  and  space  exploration, 

To  elaborate  our  theoretical  position,  we  began  with  a  rather  standard  view  of  tne  ■■ 
tational  machinery  for  early  vision,  derived  from  a  Hubel/Wiesel-like  system  of  cor.  ,3  ■ 
columns,  in  which  direct  measurements  of  image  features  were  available.  We  now  are  Peg  •  - 
to  elaborate  the  functional  circuitry  that  must  underlie  these  computations  for  pragmatic  ail/ 
successful  systems.  There  is  far  more  to  vision  than  designing  local  operators;  rather  one  must 
couple  the  design  of  operators  to  the  inference  of  image  features  by  computational  networks  to 
achieve  pragmatically  successful  systems. 

In  particular,  we  began  with  the  specific  goal  of  designing  operators  for  measuring  the  curva¬ 
ture  of  visual  features  (namely,  lines  and  edges)  from  the  information  initially  available  in  cortical 
orientation  hypercolumns,  This  was  necessary  to  support  the  neural  networks  that  we  were  build¬ 
ing  for  curve  and  edge  detection.  The  columns  were  populated  with  linear  models  of  simple  cells 
such  as  Gabor  functions  or  differences  of  Gaussians,  and  these  fed  into  standard  Hochstein  models 
of  complex  cells.  Our  insight  was  that  a  non-linear  (rectified)  combination  of  the  responses  of  cells 
with  similar  orientation  preferences,  but  with  differently  sized  receptive  fields,  could  provide  the 
orientation  signal.  This  was  substianted  by  both  simulation  and  electrophysiology.  An  important 
consequence  of  these  studies  was  the  discovery  of  a  relationship  between  endstopping  and  cur¬ 
vature  estimation,  an  observation  that  has  now  changed  the  way  many  physiologists  functionally 
interpret  the  visual  cortex. 

In  addition  to  these  theoretical  concerns,  our  laboratory  is  also  very  concerned  with  pragmatic 
computer  vision,  so  we  naturally  tried  to  apply  these  ideas  to  realistic  images  as  well.  However,  w» 
found  the  model  did  not  work  for  complex,  applied  tasks  (e.g.,  those  that  arise  in  biomedicine) 
The  problem  was  not  with  the  conceptual  ideas,  but  rather  with  the  shortcomings  of  the  initial, 
linear  operators.  When  multiple  curves  appeared  nearby,  or  when  one  object  obscured  another, 
estimates  of  orientation  were  wrong  and  the  entire  system  failed.  And  no  subsequent  network 
structuring  was  able  to  repair  it, 

We  thus  turned  our  attention  to  getting  better  initial  estimates,  and  to  uncovering  the 
structural  conditions  necessary  to  define  them,  We  are  now  developing  this  into  an  entire  theory  of 
"logical/linear  operators",  In  particular,  we  have  developed  a  calculus  for  the  logical  combination 
of  local  evidence  from  subunits  in  which  support  is  accumulated  linearly,  but  incompatible  evidence 
enters  nonlinearly.  This  led  to  a  class  of  operators  that  appears  linear  for  one  class  of  stimuli 
but  markedly  nonlinear  for  others;  hence  the  reason  for  calling  them  logical/linear  operators. 
They  exhibit  dual  advantages:  they  are  considerably  more  stimulus-specific  than  purely  linear 
operators,  while  more  robust  to  incidental  stimulus  variation  than  logical  operators,  Curiously, 
the  viability  of  such  operators  as  models  of  visual  cortical  neurons  (e.g.  simple  cells)  is  also  being 
examined,  although  it  requires  stimuli  rather  different  from  the  bars,  gratings,  and  edges  typically 
employed.  In  short,  the  non-linearities  inherent  in  logical/linear  operators  would  not  appear  in 
standard  physiological  tests,  We  believe,  however,  that  they  will  appear  if  more  complex  stimuli 
were  used, 

The  larger  context  of  our  work,  into  which  this  project  fits,  is  the  construction  of  theories 
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Together  these  results  ir«  crystalizing  a  new  perspective  on  early  visual  processing,  as  indi¬ 
cated  by  the  following  publications  that  acknowledge  AFOSR  support,  Three  graduate  student! 

have  also  been  supported  by  this  grant:  Allan  Dobbins  (McGill),  Lee  Iverson  (McGill),  and  Erica 

Strumpf  (U,8,C,),  While  we  are  proud  of  our  accomplishments  during  this  period,  we  also  believe 

that  much  remains  to  be  dene,  And  we  feel  that  we  are  in  an  excellent  position  to  con'/nce  : 

Allman,  J . ,  and  Zucker,  S  W  ,  Cytochrome  oxidase  and  functional  coding  in  primate  i'." 

An  hypothesis,  Cold  Spring  Harbor  Symp,  Quant,  Biology  1990,  55,  979  •  9 42 

Kimia,  B,,  Tannenbaurn,  A,,  and  Zucker,  S.W,,  On  the  evolution  of  curvej  via  a  fun  /, 
curvature,  1;  The  classical  case,  J-  Mathematical  Analysis  and  Application,  to  be  publish-.1 : 

David,  C,,  and  Zucker,  S,W„  Potentials,  valleys,  and  dynamic  global  coverings,  Int,  J.  Compute' 
Vision,  1990,  5,  219  •  238, 

Miller,  D,,  and  Zucker,  S,W,,  Efficent  simplex-like  methods  for  equilibria  of  nonsymmetric  analog 
networks,  Neural  Computation,  to  be  published, 

Breton,  P„  Iverson,  l.,  Langer,  M,,  and  Zucker,  5,W,,  Shading  Flows  and  Scenel  Bundles:  A 
new  approach  to  shape  from  shading,  Second  European  Conference  on  Computer  Vision, 

accepted, 

Miller,  D,f  and  Zucker,  S.W,,  A  Turing-tractable  model  of  computation  for  nonsymmetric  analog 
networks,  Snowbird  Workshop  on  Neural  Computing,  accepted, 

Dobbins,  A„  and  Zucker,  S.W,,  Mein  field  theory  and  MT  neurons,  Proc.  IEEE  Int.  Symposium 
on  Systems,  Man,  and  Cybernetics,  Boston,  Ma,,  1989, 

David,  C.,  and  Zucker,  S.W.,  Computing  global  coverings,  in  Proc.  SPIE/SPSE  Symposium  on 
Electronic  Imaging:  Curves  and  Surfaces  in  Computer  Vision  and  Graphics,  Santa  Clara, 
CA,  Feb.,  1990. 

Kimia,  B.,  Tannenbaurn,  A.,  and  Zucker,  S,W„  Toward  a  computational  theory  of  shape:  An 
overview,  Proc.  ltl  European  Conf.  on  Computer  Vision,  Antibes,  France,  April,  1990.  0 
Faugeras  (ed.),  Lecture  Notes  in  Computer  Science,  427,  Springer  Verlag,  New  York, 

Zucker,  S,W,,  Iverson,  L,,  and  Dobbins,  A,,  On  the  neural  computations  underlying  curve  detec¬ 
tion,  Neural  Networks  for  Sensory  and  Motor  Systems,  Dusseldorf,  March,  1990, 

Dobbins,  A„  Iverson,  L.,  and  Zucker,  S,W„  A  logical/linear  model  of  cortical  subunit  interac¬ 
tion,  Annual  Meeting,  The  Association  for  Research  in  Vision  and  Opthalamology  (ARVO), 
Florida,  April,  1990, 

Dobbins,  A„  Zucker,  S,W„  and  Cynader,  M„  A  mean  field  model  of  optic  flow  by  MT  neurons, 
2&h  Annual  Meeting  of  the  Society  for  Neuroscience,  1990, 

Zucker,  S.W,,  Dobbins,  A,,  and  Iverson,  L„  On  the  computational  neurobiology  of  curve  detection, 
Proc.  British  Machine  Vision  Conf,,  Oxford,  Sept,,  1990, 

Kimia,  B„  Tannenbaurn,  A,,  and  Zucker,  S,W„  A  conservation-minded  approach  to  shape,  Proc. 
IEEE  Conf.  on  Intelligent  Control,  Philadelphia,  1990, 

Dobbins,  A,,  Iverson,  L„  and  Zucker,  S,W,,  A  mean  field  model  of  optic  flow  estimation,  Annual 
Meeting,  The  Association  for  Research  in  Vision  and  Opthalamology  (ARVO),  Florida,  April, 
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